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(57) A method and system for using indicated text 
to automatically query a search engine for a database 
and return Images that are relevant to the Indicated text. 
At least five integrated components are employed by a 
client program to get, view and use a retumed Image. A 
user interface component provides for automatically 
creating search criteria for a query from the indicated 
text and inserting a thumbnail or higher quality image 
into an open document. A metadata results component 
provides for managing the storing, parsing and display- 
ing of the metadata results retumed from a query of the 
database of images. A navigation component provides 
for managing user interface navigation elements that 
are employed to control the display of the results 
(thumbnail images) of the query, e.g., play, stop, back 
and next controls. An image binary data component pro- 
vides for managing the display of retumed images and 
managing the storage of the binary image data for the 
thumbnail and higher quality images. Also, a network 
component provides for detecting the presence of a net- 
work and its settings and managing the transfer of data 
across the network such as the Intemet and an intranet. 
Additionally, a qualification engine may be used to de- 
termine the context of the indicated text. An electronic 
commerce service may be used to purchase higher 
quality versions (improved resolution and/or color) of the 
thumbnail images. 
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Description 

Field of the Invention 

[0001] This application relates generally to obtaining 
inoages fronn a database, and, more specifically, to re- 
turning chosen images from the database that are rele- 
vant to indicated text In a document. 

Background of the Invention 

[0002] In the past, when a user of an application pro- 
gram wished to add an image to a file created by that 
application, the user manually searched a database of 
images or surveyed a relatively large group of images 
organized into predetermined categories. Alternatively, 
the user might be required to pre-purchase access to a 
database of images prior to searching for an image rel- 
evant to the content of the document. In either case, the 
user would be required to indicate the relevant image 
by manually comparing a set of preselected iniages to 
the content of a document. 

[0003] Therefore, it is desirable to provide a simple 
and intuitive facility for enabling a user to choose an im- 
age relevant to the content of a document. The facility 
could be a stand alone application program or integrated 
with an existing application program, e.g., a word proc- 
essor, slide show presentation or email program. It is 
also desirable to provide a facility that enables the user 
to purchase a chosen image and indicate the quality and 
options associated with the purchased innage. Also, the 
facility should enable the purchased image to be viewed 
and printed with any widely available application pro- 
gram. 

Summary of the Invention 

[0004] In accordance with the present invention, a 
method for searching a database to obtain an object that 
is related to an indicated object, comprising: (1) auto- 
matically creating a query related to the indicated object, 
the query having a data structure that is recognizable 
by a search engine for the database; (2) providing the 
query to the search engine, the search engine searching 
the database for at least one object that is related to the 
indicated text; (3)retuming the result from the query of 
the database, the result indicating when at least one ob- 
ject is related to the indicated text; and (4) producing a 
display of a related object, so that the related object may 
be associated with the indicated object. The indicated 
object and the related object may include data such as 
video, picture, sound, and text. Also, a qualification en- 
gine may be used to determine a context of the indicated 
object. The query for the database can be automatically 
created based on the context of the indicated object Ad- 
ditionally, the related object may be purchased over a 
network. When complete financial information provided 
over a network to an electronic commerce service, the 



sen/ice can enable the purchase of the related object. 
[0005] In accordance with another embodiment of the 
present invention, a method provides for obtaining an 
image from a database for insertion into a document, 
s the image being related to indicated text, comprising: 
(1 ) enabling an automatic creation of a query related to 
the indicated text, the query having a data structure that 
is recognizable by a search engine for the database; (2) 
providing the query to the search engine, the search en- 
10 gine searching the database for an image that is related 
to the indicated text; (3) returning the result from the 
query of the database, the result orKticating when at least 
one image is related to the indicated text; (4) enabling 
the display of at least one image indicated by the result 
IS from the query, a displayed image being selectable for 
insertbn into the document; and (5) displaying a related 
image that is inserted into the document, so that the re- 
lated image is associated with the indicated text In the 
document. 

[0006] In accordance with other aspects of the other 
embodiment of the present rnventk>n, the method pro- 
vides for employing a user interface component to au- 
tomatically create the query for the database, the crea- 
tion of the query being related to the indicated text. The 
user interface component enables the indicated text to 
be edited and chosen in the document. Ttie indicated 
text may be chosen from within an application program 
that provides the document. A command can be chosen 
to create the query for the indicated text, the command 
may include a selectbn in a context menu, drop-down 
menu, pop-up window, dialogue box, toolbar and hot 
key The indicated text may be chosen in another appli- 
cation program, e.g., a clipboard program, that is sepa- 
rate from the applicatkjn program that provides the doc- 
ument. 

[0007] In accordance with yet other aspects of the oth- 
er embodiment of the present invention, the method pro- 
vides for enabling a qualification engine to detemnine a 
context of the indicated text within the document. The 
context of the indicated text may be emptoyed by the 
user interface component to automatically create the 
query for the database. The determined context of the 
indicated text may include text, template, sourrd, video, 
pk:ture, use and user preference. 
[0008] In accordance with still other aspects of the 
other embodiment of the present invention, the search 
engine may be a local search engine that employs the 
query to search for related images on a kx^al drive that 
includes the database. The local drive nnay include a 
hard disk, floppy disk, tape drive. DVD and CD-ROM. 
The query of the database may include an XML data 
structure. Also, a network component may be emptoyed 
to communicate with the search engine over a network 
that can include an intranet and the Intemet. Addition- 
ally, a result component may be emptoyed to manage 
the result of the query returned by the search engine, 
the result component enabling the parsing, storing and 
display of the result of the query. The result may be pro- 
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vided in a metadata format. 

[0009] In accordance with other aspects of the other 
embodiment of the present Invention, the method pro- 
vides for requesting at least one related image from the 
database that is indicated by the resuJt of the query, the 
related Image being provided as binary image data. An 
Innage component may be provided to manage the stor- 
age and display of binary Image data 
[001 0] I n accordance with yet other aspects of the oth- 
er embodiment of the present invention, a navigation 
component may be employed to provide a control for 
displaying at least one related irhage. The navigation 
component enables a slide show display of a group of 
related Images that are indicated in the result from the 
query. The navigation component enables the slide 
show to display other groups of related Images that are 
Indicated In the result from the query. The navigation 
component may ir^clude a control for determining a pe- 
riod of time that each related image is displayed in the 
slide show. Also, the navigation component can Include 
a control for Indicating the number of the related image 
in the group that is currently displayed in the slide show. 
Additionally, the navigation component may include a 
control for indicating each group of related images that 
are currently displayed In the slide show. 
[001 1 ] In accordance with yet other aspects of the oth- 
er embodiment of the present Invention, a help facility 
may be launched in response to an input and provide 
an explanation of a chosen topic. Also, a thumbnail of 
the related image may be displayed. Additionally, a dig- 
ital watermark may be Included with the related image 
which provides Information related to the content of the 
related image. Furthermore, the query can include a 
cookie that rr^y provide Information related to a user. 
[0012] In accordance with other aspects of the other 
embodiment of the present invention, the method pro- 
vides for: (1 ) indk:atlng a higher quality image of the re- 
lated Image is to be purchased over a network; (2) pro- 
viding financial information to an electronic commerce 
service over the network, the electronic commerce sen^- 
ice enabling the purchase of the higher quality image 
when the financial information Is complete; and (3) in- 
serting the purchased higher quality Image Into the doc- 
ument so that the higher quality image is displayed with 
the Indicated text. An electronic shopping cart may be 
empbyed to store a potential purchase of the higher 
quality image. A parameter and use of the higher quality 
image may be indicated. 

[001 3] In accordance with yet other aspects of the oth- 
er embodiment of the present Invention, each time Indi- 
cated text Is employed to create the query, the method 
may automatically save the Indicated text as an entry in 
a history file and enable at least one of the entries in the 
history file to be chosen for the currently Indicated text. 
Also, a URL may be embedded in the related image 
pasted into the document. The URL can provide a link 
to information associated with the related image. Addi- 
tionally, the method can automatically connect to the 



search engine for the database when the search engine 
is remotely tocated from the indicated text. Also, when 
no indicated text Is provided, a word that is closest to a 
cursor may be determined and empbyed as the indlcat- 
s ed text. 

[001 4] In accordance with another embodiment of the 
present invention, a method for electronically purchas- 
ing an image over a network, comprising: (1 ) automati- 
cally creatirig a query related to an indicated object, the 
10 query having a data structure that is recognizable by a 
search engine for the database; (2) providing the query 
to the search engine, the search engine searching the 
database for at least one image that Is related to the 
Indicated object; (3) returning the result from the query 
of the database, the result Indicating when at least one 
image is related to the ind bated object; (4) Indicating a 
related Image Is to be purchased over the network, fi- 
nancial information being provided to an electronic com- 
merce servbe that enables the purchase of the related 
Image when the financial infornnatlon is complete; and 
(5) when the related innage Is purchased, enabling a dis- 
play of the related image, so that the related image may 
be associated with the Indbated object. 
[001 5] In accordance with other additional aspects of 
the present invention, a system which implements sub- 
stantially the same functbnality in substantially the 
same manner as the methods described above is pro- 
vided. 

[001 6] In accordance with yet other additional aspects 
of this Inventbn, a computer-readable medium that in- 
cludes computer-executable instructions may be used 
to perform substantially the same methods as those de- 
scribed above Is provided. 

Brief Descriptbn of the Drawings 

[001 7] The foregoing aspects and many of the attend- 
ant advantages of this invention will become more read- 
ily appreciated as the same become better understood 
by reference to the folbwing detailed description, when 
taken in conjunctbn with the accompanying drawings, 
wherein: 

FIGURE 1 illustrates indicated text and a context 
menu in a document displayed in a graphical user 
Interface for an exemplary email program; 
FIGURE 2 shows an exemplary pop-up window for 
an Imaglzer program; 

FIGURE 3 illustrates an exemplary pop-up window 
for options associated with the Imagizer program; 
FIGURE 4 shows a graphical image disposed in the 
document displayed in the graphical user Interface 
for the exemplary ennail program; 
FIGURES 5A-5B Illustrate a flowchart of the logic 
for obtaining an image that Is relevant to Indicated 
text; 

FIGURE 6 shows a schematic oven^iew of a client 
program that Includes five separate components for 
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performing functions related to obtaining images 
that are relevant to the incfcated text; 
FIGURE 7 illustrates a schematic overview of com- 
munication between the client program and a server 
program for obtaining images that are relevant to 
the indicated text; 

FIGURE 8 shows an exemplary computer system 
for implementing the client program; and 
FIGURE 9 is a schematic overview illustrating the 
system architecture for implementing the present 
invention over the Internet 

Detailed Description of the Preferred Embodiment 

[0018] The present invention enables a user to 
choose an inr^ge from a group cf iniages that are de- 
termined to be the most relevant for indicated text. The 
present invention uses indicated text to find relevant im- 
ages by automatically generating a query based on the 
indicated text for a database of images and displaying 
the query results as thumbnail images in a slide show. 
The user may insert any of the thumbnail images into a 
document or purchase a higher quality version of an in- 
dicated thumbnail image for use in the document. An 
electronic shopping carl may be provided for temporarily 
storing several higher quality images that may be pur- 
chased individually or as a group at a later date. 
[0019] It is envisioned that the functionality of the 
present invention may be provided in different ways, in- 
cluding a stand-atone application program and an add 
on (extension) facility incorporated into an application 
program. For example, the present invention may be im- 
plemented as an executable file (EXE), as a Visual Ba- 
sic control (VBX) or as a dynamic linked library (DLL). 
Also, the application programs that work with the 
present invention may include email, spreadsheet, word 
processor, slide show presentation, browser and editor. 

Graphical User Interface 

[0020] FIGURE 1 illustrates an exemplary graphical 
window 100A for an email application program that in- 
corporates an Imagizer facility tor implementing the 
functionality of the present invention. The Imagizer fa- 
cility may be integrated into the email application pro- 
gram in several ways, including a DLL or a separate add 
on program. 

[0021] In a document window 102, the sentence 
(string) 'Do you want to learn how to sail" is positkxied 
in the upper left hand corner The word "sail" 106 is high- 
lighted (indicated) in the sentence and a context menu 
1 04 is disposed adjacent to the indicated word. Alterna- 
tively, the context menu 104 nnay be disposed adjacent 
to a cursor However, in this example, the context menu 
104 for the word "sail" 106 is displayed when a button 
on a user interface pointing device, e.g., a mouse, is 
selected. The context menu 1 04 lists several commands 
including cut, paste, font and imagizer When an imag- 



izer command 108 is selected, a pop-up window 1 12 en- 
titled "imagizer" is displayed by the Imagizer facility, as 
shown in FIGURE 2. Although not shown, it is envi- 
sk>ned that the imagizer command may be selected as 
5 an element on a toolbar and as a menu item in a drop- 
down menu. Additionally, when no text is indicated, the 
Imagizer facility may determine the nearest word to the 
cursor and employ that word as the indicated text for 
searching the database of images. 
[0022] In FIGURE 2, the pop-up window 112 illus- 
trates a thumbnail image 1 1 0A of a sailboat. The thumb- 
nail image 11 OA is disposed in the upper portion of the 
pop-up window 112 and the thumbnail image is the cur- 
rently displayed image in a slide show for a group of im- 
ages relevant to the indicated word "sail" 106. A slide 
control 118 is disposed in the tower portion of the pop- 
up window 1 1 2 and the control is animated to graphically 
illustrate the progressive display of each thumbnail im- 
age in the slide show. Additionally, the stkie control 118 
may be selected to cause the slide show to move to the 
display of a particular thumbnail image. A button 114, 
entitled "Purchase HQ Image," is displayed adjacent to 
the thumbnail inr>age 11 OA in the upper right hand por- 
tion of the pop-up window 112. As described in greater 
detail below, the selection of the button 114 causes an 
electronic commerce transactton to be initiated for the 
purchase of a higher quality (higher resolution) version 
of an indicated thumbnail image. 
[0023] Another button 1 1 6, entitled "Paste Image Into 
Document," is displayed adjacent to the thumbnail im- 
age 11 OA in the upper right hand portion of the pop-up 
window 112 and directly betow the button 114. As de- 
scribed in greater detail below, the selection of the but- 
ton 1 16 causes an indicated image, e.g., a currently dis- 
played thumbnail image or a purchased higher quality 
version, to be pasted into the document adjacent to the 
indicated word "sail" 106. 

[0024] A button 115, entitled "Send Picture Mail," is 
displayed adjacent to the thumbnail image 11 OA in the 
uppermost right hand portion of the pop-up window 112, 
directly above button 114. The selection of the button 
1 1 5 causes an email facility to create and send an email 
message that includes the indicated image from within 
the Imagizer facility. Although not shown, the email fa- 
cility enables the user to enter an email destinatton and 
text to accompany the indicated image. 
[0025] Another button 120, entitled ■«Back," is dis- 
played in the lower left quarter of the pop-up window 1 1 2 
and below the slider control 118. Directly to the right of 
the button 120, another button 122, entitled "Play," is 
disposed and a button 124, entitled 'Next»,'' is posi- 
tioned to the right of the button 122. The three button 
120, 122 and 124 enable a user to navigate the slide 
show by selectively controlling the play, advance and 
replay of the thumbnail images. A button 1 26, entitled 
"More," is displayed to the right of the button 124 
('Next»") and the selection of the button 126 causes 
another pop-up window 128, entitled "More," to be dis- 
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played, as shown in FIGURE 3. 
[0026] Although not shown, it is envisioned that the 
functionality of the ■«Back' button 120 and the 
■Next»' button 1 24 nnay be incorporated into a horizon- 
tal scrollbar that includes a pointer. The horizontal scroll- 5 
bar could include separate buttons disposed at oppos- 
ing ends of the scrollbar and the selection of a button 
would enable a forward or backward r^anngation of the 
group of thumbnail images. Also, the pointer might move 
along the scrollbar as each thumbnail ima^ in the group io 
is sequentially displayed. By selecting the position of the 
pointer on the scrollbar, the user coukJ navigate through 
the group of thumbnail images. 
[0027] FIGURE 3 shows the "More" pop-up window 
1 28 with a text entry box 1 30 positbned in the upper left is 
hand quarter of the pop-up window. A button 132, enti- 
tled "Search, " is disposed adjacent to the right hand side 
of the text entry box 1 30. When text is entered in the box 
130 and the search button 132 is selected, this action 
causes a database to be searched for images relevant 
to the entered text, which is discussed in greater detail 
below. Also, a group of relevant images returned from 
the database are displayed in a slide shoM^ as shown in 
the pop-up window 112 in FIGURE 2. 
[0028] A button 134. entitled "< Back 50," is posi- 2S 
tioned directly betow the text box 1 30 in the bottom left 
hand of the pop-up window 1 28 and another button 1 36, 
entitled "Next 50 >." is positioned directly below the 
search button 132 in the bottom right hand of the pop- 
up window. Also, a view box 135, entitled "hiow View- 30 
ing", is positioned horizontally between the buttons 134 
and 136, whk;h displays reference numbers for the 
group of relevant thumbnail images thai are currently 
displayed in the slide show. 

[0029] The slide show displayed in pop-up window 3S 
112 uses a predetermined number (group) of thumbnail 
images that are relevant to the indk:atecVentered text. 
However, the present invention enables more than one 
group of relevant images to be viewed in the slide show 
by using the buttons 1 34 and 1 36 to navigate the display 40 
of different groups of Images. When the '< Back 50" but- 
ton 1 34 is selected, this action will cause a prevbusly 
displayed group of fifty relevant thumbnail tnr^ges to be 
used in the slide show. Similarly, when the "Next 50 >" 
button 136 is selected, the next group cf fifty relevant ^ 
thumbnail images will be displayed in the slide show 
[0030] Alternatively, a seamless slide show display 
(no 50 image pagination) of the entire group of thumb- 
nail images may be provided. In this case, the buttons 
1 34 and 1 36 would not be provided in this "More" pop- so 
up window 128. 

[0031] A slide control 138 is disposed in the bottom 
portk)n of the pop-up window 1 28. The manipulation of 
the slider control 1 38 will cause an increase or decrease 
in the anrxjunt of time each thumbnail image is displayed ss 
in the slide show. Also, a button 140, entftled "help," is 
disposed below the slide control 138 in the tower left 
hand of the pop-up window 128. The selection of the 



help button 1 38 causes a help facility to display infor- 
mation regarding a chosen topic in an index. Ancrther 
button 142, entitled "About Imagizer," is positioned be- 
tow the sIkJe control 1 38 in the lower right hand of the 
pop-up window 128. The selection of the button 142 
causes the display of informal ton regarding the version 
of the Imagizer facility and other related matters. Also, 
a button 131, entitled "About Indicated Image", is posi- 
tioned in the upper left most portion of the pop-up win- 
dow 128, Ttie selection of the button 131 causes the 
display of related infomnatton that is included in meta- 
data associated with an indicated image. For example, 
the Imagizer facility may display related information that 
includes an image identification number, author, original 
date that image of subject nnatter was captured, tocation 
and descriptive title. It is also envisioned that this infor- 
mation may be automatically displayed by the Imagizer 
facility along with a display of each related image. 
[0032] FIGURE 4 illustrates another exemplary win- 
dow 1 0OB for the applicatton program that incorporates 
the Imagizer facility. In the document window 102, a 
higher quality (version) image 11 OB of a sail boat is past- 
ed into the open document window adjacent to the word 
"sail" 106 in the right hand portion of the window. This 
example shows the results when a higher quality version 
of a thumbnail innage is purchased and pasted into an 
open document. 

[0033] Additionally, an embedded link (URL) may be 
automatically added to a pasted image. When the past- 
ed image is selected, the embedded URL could provide 
a link to an information page associated with the pasted 
image on a remotely located sender. 
[0034] Generally, a thumbnail image may have a res- 
olution of 100 X 100 pixels. However, the higher quality 
versions of the relatively low resolution thumbnail image 
can be provided at increasingly greater resoluttons for 
a correspondingly larger cost, e.g., a 256 x 256 pixel 
image might be free, a 640 x 640 pixel image could cost 
$5.00 and a 1 024 x 1 024 pixel image might cost $20.00. 
It is en vis toned that the actual cost of the innages could 
be controlled/changed by a server that enables the Im- 
agizer facility to access a database of images. 
[0035] FIGURES 5A and 5B show an overview 1 44 of 
the logto performed by the present inventton to get, in- 
dicate and insert a relevant image into an open docu- 
ment. In FIGURE 5A, nrxjving from a start btock, the logto 
steps to a btock 146 where the Innagizer facility is acti- 
vated. In this example, the Imagizer facility is activated 
from a context menu. However, it is envisioned that the 
Imagizer facility may also be launched from a pop-up 
window, drop-down menu, dialogue box and hot key 
Additionally, in this example, the Imagizer facility acts 
as a client providing a query to a search engine server 
that can search a database for relevant innages. 
[0036] Ftowing to a decision block 148, a determina- 
tion is made whether text is indicated. If false, the logto 
moves to a block 156 where the "options' pop-up win- 
dow 128 for the Imagizer facility is displayed as shown 
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in FIGURE 4. Advancing to a decision block 158, the 
logic determines whether indicated text is provided in 
the text entry box 1 30 of the options pop-up window 1 28. 
If false, the logic loops back to the block 156 until the 
pop-up window 1 28 is closed or the indicated text is pro- 
vided in the text entry box 130. However, when the de- 
termination at the decision bkx:k 158 is true, the logic 
will move to a decision block 150. Also, if the determi- 
nation at the decision block 1 48 was true, the logic would 
have moved to the decision bkx;k 150. Additkxially, the 
text could be indicated in a separate clipboard program; 
and although not shown, the history of previously indi- 
cated text may be saved and reused as the currently 
indicated text. 

[0037] At the decision block 150, a determination is 
made whether the I magizer facility can connect directly 
to the Internet. If false, the togic rrKtves to a decision 
block 160 where a determinatkjn is made whether the 
I magizer facility can connect to the Internet through an 
intranet connection. If the detenmination at the decision 
block 160 is false the logic will move to a decisbn block 
1 62 where a determination is made whether a kx^al serv- 
er for searching a database of images is available. The 
Imagizer facility can determine if a kx:al drive contains 
a database of images that might be relevant to the indi- 
cated text. Also, the Imagizer facility may prompt the us- 
er to indk^ate the path to a local image database. 
[0038] If the determlnatbn at the decision block 162 
is negative, the logic moves to a decision block 175, as 
shown in FIGURE 5B. However, if the determination at 
the decisk>n block 162 is affirrmtive, the k>gic steps to 
a block 163 and the InDagizer facility connects with the 
local image database. The logic advances to a block 1 52 
where the Imagizer facility creates a request (query) for 
relevant images based on the indicated/entered text. 
The request is sent to a local search engine server to 
return metadata results from a data base of images on 
a local drive, e.g., a hard disk, floppy disk, tape drive, 
digital versatile disc (DVD) drive and compact disc read 
only memory (CD-ROM) drive. 
[0039] Alternatively, when the determination at the 
decisbn block 150 is true, the k>gic moves to the block 
1 52 where the Imagizer facility connects through the In- 
ternet to another search engine sender that returns 
metadata results from a remotely located image data- 
base. Also, if the determination at the deciskxi block 1 60 
is true, the logk: steps to a block 1 61 where an autocon- 
nect facility is invoked to connect the Imagizer facility to 
the remotely located image database. The autoconnect 
facility can determine if a proxy server is available for 
connecting across the Internet to the remotely located 
image database. Alternatively, the autoconnect facility 
may initiate a dial-up telephone connection to an indi- 
cated Internet Service Provider (ISP) to create a con- 
nection to the remotely kx:ated image database. The au- 
toconnect facility may prompt the user to provide a new 
telephone number for the ISP when a previously provid- 
ed telephone number is not available. 



[DO40] The logk: moves from the bkx:k 161 to the 
block 152 where the Innagizer facility connects with the 
other search engine server. An. important aspect of the 
present invention is sending a query and receiving re- 
5 suRs over a network without having to launch a separate 
applk;ation program such as a browser. Additionally, the 
query nnay employ an extensible markup language 
(XML) data structure to communicate over a network 
with a search engine for searching an image database 
10 such as the CORBIS CORPORATION'S CORBIS PIC- 
TURE EXPERIENCE at http://www.corbis.com. 
[0O41] The logk; flows to a decision block 1 54 and de- 
termines if the metadata results from the query were re- 
turned. If false, the logic jumps back to the bkx:k 156 
where substantially the same bgic discussed above is 
repeated. However, if the determination is true, then the 
togic moves to a block 164 as shown in FIGURE 5B and 
where the first group of binary image data is placed in 
a local cache managed by the Imagizer facility. Although 
the metadata results may kJentify hundreds of relevant 
images, the present invention provides for downloading 
orte group at a time of the most relevant images for dis- 
play in a slide show. For this example, the first group 
downloaded includes fifty of the nrK>st relevant thumbnail 
images as binary image data. However, in another em- 
bodiment, the binary image data for all of the relevant 
images klentified by the metadata results could be 
downloaded for display in the slide show. 
[0042] The logic steps to a block 166 where the first 
group of thumbnail images are displayed in the slide 
show such as illustrated in pop-up window 128 of FIG- 
URE 2. Moving to a decision block 1 68, a determination 
is nrtade as to whether one of the thumbnail images in 
the sikie show is indk:ated for inserting (pasting) into the 
open document. If affirmative, the logk: advances to a 
btock 174 where the indicated thumbnail image is past- 
ed onto the open document. Next, the togic flows to the 
decision block 175 where a determinatton is made 
whether a user has chosen to close the Innagizer facility. 
If yes, the togk; steps to an exit btock and tenminates. 
However, if the determination at the decision block 1 75 
is no, the togto will move to a block 177 where new in- 
dcated text is provided. Next, the logk; loops back to the 
decision btock 148 where substantially the same logic 
discussed above is be repeated. 
[0043] Alternatively, when the determination at the 
decision btock 168 is negative, the logic steps to a de- 
cision block 170 where a determination is made as to 
whether the purchase of a higher quality version of the 
thurrtbnail image is indtoated. If false, the logic moves 
to a deciston block 176 where a determinatton is made 
as to whether a request for the next group of thumbnail 
images is provided. If negative, the logb moves to the 
decision btock 175 where substantially the same logk: 
discussed above is repeated. However, when the deter- 
minatton at the decision block 176 is positive, the logic 
returns to the btock 166 where the next group of 50 
thumbnail images are displayed In a slide show and sub- 



20 



25 



30 



35 



40 



45 



SO 



6 



11 



EP1 077 414 A2 



12 



stantially the same logic discussed above is repeated. 
[0044] Also, when the determination at the decision 
block 170 is positive, i.e., the purchase of a higher qual- 
ity image is indicated, the logic moves to a block 172 
where a browser program is launched to connect to an 
Internet based web site for performing an electronic fi- 
nancial transaction service that enables the purchase of 
a higher quality version of the indicated thumbnail im- 
age. The logic flows to a block 1 78 where the purchased 
higher quality image is inserted into the open document. 
Next, the logic advances to the block 166 where the 
same group of images are displayed in the slide show 
and substantially the same logic discussed atx>ve is re- 
peated. 

[0045] Although not shown, different parameters of 
the higher quality image may be chosen such as reso- 
lutbn and cotor. Also, the use of the higher quality image 
may be identified, e.g., personal, work and publicatbn. 
The chosen parameters and use may be employed to 
determine the cost of purchasing the higher quality ver- 
sion of the indicated thumbnail image. 
[0046] FIGURE 6 illustrates five integrated compo- 
nents that may be empbyed by an Imagizer facility 180 
to get, view and use an indicated image. A graphical us- 
er interface component 182 provides for automatk:alty 
creating search criteria for a query from the indicated 
text and inserting a thumbnail or higher quality image 
into an open document. A metadata results component 
188 provides for managing the storing, parsing and dis- 
playing of the metadata results returned from a query of 
the database of images. A navigatbn component 190 
provides for managing user Interface navigation ele- 
ments used to control the display of the results (thumb- 
nail images) of the query, e.g., play stop, back and next 
controls. An image binary data component 1 86 provides 
for managing the display of returned images and man- 
aging the storage of the binary image data for the thumb- 
nail and higher quality images. Also, a network compo- 
nent 1 84 provides for detecting the presence of a net- 
work and its settings and managing the transfer of data 
across the network such as the Internet and an intranet. 
The network component nr^ay employ a transmisskvi de- 
vice to automatically connect to the search engine for 
the database, e.g., a cable modem, network interface 
card, telephony modem, satellite dish and radk> trans- 
ceiver. 

[0047] In one embodiment, the network component 
1 84 may launch a separate browser program to manage 
the transfer of data over a network. In another embodi- 
ment, the network component 184 will manage the data 
transfer from within the Imagizer facility without having 
to launch the browser program. The network component 
184 could send the query to a remotely located search 
engine, obtain the result of the query and downtoad 
thumbnail images indk:ated by the query result. In yet 
another embodiment, a browser program coukJ be 
launched by the network component 184 for managing 
the transfer of financial data to purchase of a higher 



quality version of an indicated thumbnail image from an 
electronic commerce server. In still another emtxxJi- 
ment, the network component may manage the transfer 
of financial data to purchase a higher quality image with- 

5 out having to launch the browser program. 

[0048] FIGURE 7 shows a graphical representation of 
an overview 202 of how data packets (information) are 
communicated between the Imagizer facility acting as a 
client 204 and a search engine acting as a server 206 

10 that receives the client's query and returns results, e.g., 
metadata, from a database of images. A time line 208 
is positioned vertbally adjacent to a graphical represen- 
tation of an exchange of requests and responses be- 
tween the client 180 and the sewer 206. Moving down 

15 the time line 208 from the top of the figure and an initial 
time value of 'zero,' the client 204 provides a "Get send- 
er ip address" request 21 0 to the server 206. In reply 
the sender 206 sends a "Server ip address" response 
21 2 to the client 204. Next, the client 204 communicates 

20 a "Get image list" request 214 to the server 206. This 
request includes a query based on the indicated text. 
Advancing down the time line 208, the server 206 com- 
municates a "Send image list" response 21 6 to the client 
204 which includes metadata results indicating a list of 

25 the images that are detemiined to be relevant to the in- 
dicated text. 

[0049] A "Get thumbnail images" request 21 8 is com- 
municated by the client 204 to the server 206 indicating 
a particular group of thumbnail images. The server 206 

30 provides a "Send thumbnail images" response 220 to 
the client 204 which includes the binary image data for 
the requested group of thumbnail images. Optk)nally, 
when the client 204 has requested a purchase of a high- 
er quality image, a "Get Purchased HQ Image" request 

35 222 will be provided by the client to the server. In reply, 
the sender 206 will provide a "Send Purchased HQ Im- 
age" response 224 to the client 204 that includes the 
binary image data for the purchased higher quality im- 
age. 

40 [0050] FIGURE 8 illustrates a schematic oven/iew 
192 of a client-server architecture for implementing the 
present invention. The client 204, server 206 and an 
electronic commerce server 200 are in communication 
with each other over a network 1 98 such as an intranet 

45 and the Internet. The server 206 is also in communica- 
tion with a database 198 that may be searched for im- 
ages relevant to a query related to indicated text provid- 
ed by the client 204. Additionally, it is envisioned that 
different types of transport protocols may be employed 

50 to communicate between the client 204, server 206, and 
electronic commerce server 208, including hyper text 
transport protocol (HTTP), file transfer protocol (FTP) 
and transmission control protocol/internet protocol 
(TCP/IP). 

55 [0051] The electronic commerce server 200 provides 
a sewice that enables the conduct of commerce in 
goods and sen/ices, with the assistance of telecommu- 
nications and telecommunications-based tools. The cli- 
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ent 204 may provide a credit card number to the elec- 
tronic commerce server 200 to purchase a higher quality 
(resolution) version of an indicated thumbnail image 
which is retrieved from the database 198 by the server 
206 and downloaded to the client. The electronic com- 
merce sender 200 may empby a digital signature and 
public/private key based encryption to complete the fi- 
nancial transaction. 

[0052] FIGURE 9 illustrates a system 10 comprising 
components of a computer suitable for executing a client 
program embodying the present invention. A processor 
12 is coupled bi-directionally to a memory 14 that en- 
compasses read only memory (ROM) and random ac- 
cess memory (RAM). ROM is typically used for storing 
processor specific machine code necessary to bootup 
the computer comprising client 10. to enable input and 
output functions, and to carry out other base aspects of 
its operation. Prior to running any application program, 
the machine language code comprising the program is 
loaded into RAM within memory 14 and then executed 
by processor 12. Processor 12 is coupled to a display 
1 6 on which the visualization of the HTML response dis- 
cussed above is presented to a user. Often, programs 
and data are retained in a nonvolatile memory media 
that may be accessed by a compact disk-read only 
memory (CD-ROM) drive, compact disk-read/wrrte 
memory (CD-R/W) drive, optical drive, digital versatile 
disc (DVD) drive, hard drive, tape drive and fk)ppy disk 
drive, all generally indicated by reference numeral 18 in 
FIGURE 9. A network interface 22 couples the proces- 
sor 1 2 to a wide area network such as the Internet. 
[0053] As noted above, the present inventk>n can be 
distributed for use on the computer system 10 as ma- 
chine instructions stored on a memory media such as a 
floppy disk 24 that is read by the floppy disk drive. The 
program would then typically be stored on the hard drive 
so that when the user elects to execute the application 
program to carry out the present invention, the machine 
instructbns can readily be toaded into menrxxy 14. Con- 
trol of the computer and selection of options and input 
of data are implemented using input devk^es 20, which 
typically comprise a keyboard and a pointing device 
such as a mouse (neither separately shown). Further 
details of the system 1 0 and of the computer comprising 
it are not illustrated, since they are generally well known 
to those of ordinary skill in the art. Additionally, although 
not shown, a computer system for the server 206 could 
be configured in substantially the same way as the com- 
puter system 10 illustrated here, albeit different in other 
ways. 

[0054] In one embodiment, an electronic shopping 
cart may be used for the temporary storage a potential 
purchase of a higher quality verskxi of at least one indi- 
cated thumbnail image. An electronic shopping cart is 
an applicatbn program that enables on-line image se- 
lectran and ordering processes. Typically, an electrons 
shopping cart is part of the interface between an Internet 
web site and its deeper infrastructure, e.g., an image 



database, allowing users to select merchandise; review 
what they have selected; make necessary modifications 
or additions; and purchase the selected merchandise. 
It is envisioned that the electronic shopping cart would 
s temporarily hold a higher quality image until the pur- 
chase could be completed at a later date when final pur- 
chasing decisions are made and/or sufficient funds be- 
come available. 

[0055] In another embodiment, the present invention 
10 may employ a separate qualification engine to assist in 
the creation of the query for the search engine to locate 
relevant images in an image database. It is envisioned 
that the qualification engine may determine the context 
of the indbated text, e.g., the qualification engine may 
IS examine the context of the indicated text within a sen- 
tence, paragraph, or entire document. Also, the qualifi- 
cation engine may empby the content of other images 
in the document and user preferences to help create the 
query. 

20 [0(^6] In one embodiment, a digital watermark may 
be included with each image. A digital watermark is a 
represented by a pattern of bits inserted into a file for an 
image. The digital watermark may be designed to be 
completely invisible or visible and the actual bits repre- 
ss senting the watermark tend to be scattered throughout 
the file in such a way that they cannot be klentified and 
manipulated. The digital watermark can withstand nor- 
mal changes to the file, such as reductbns from lossy 
compression algorithms. It is envisioned that a digital 
30 watermark may be employed to identify copyright infor- 
matbn and provide information related to the content of 
the image. 

[0057] In yet another embodiment, a cookie identify- 
ing user preferences may be included with the query that 

35 is sent to the search engine. The search engine coub 
use the cookie informatk>n to help provide metadata re- 
sults that are related to the user preferences. Also, the 
cookie information may provide a user-specific set of re- 
sults, such as a list of innages that were previously past- 

40 ed into documents, a history of prevbusly ordered/pur- 
chased images, and a user-organized collection of im- 
ages, e.g., a portfolio. 

[0058] In yet another embodiment, the present inven- 
tion may be employed to automatically generate a query 

^ for an indicated object and return results from a search 
engine that identifies any object that is related to the in- 
dbated object. The indbated object and the related ob- 
ject may include different types of data comprising text, 
pictures, video, and sound. For example, an indicated 

50 text object could be used by the present inventbn to 
cause a search engine to return different related objects 
that include text, video and sound (audio). 
[0(^9] While the preferred embodiment of the inven- 
tion has been illustrated and described, it will be appre- 

55 ciated that various changes can be made therein without 
departing from the spirit and scope of the invention. 
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Claims 

1 . Method for searching a database to obtain an object 
that is related to an indicated object, comprising: 

5 

(a) automaticaDy creating a query related to the 
indicated object, the query having a data struc- 
ture that is recognizable by a search engine for 
the database; 

(b) providing the query to the search engine, io 
the search engine searching the database for 

at least one object that is related to the indicat- 
ed text; 

(c) returning the result from the query of the da- 
tabase, the result Indicating when at least one is 
object is related to the indicated text; and 

(d) producing a display of a related object, so 
that the related object may be associated with 
the Indicated object. 

20 

2. The method of Claim 1 , wherein the indicated object 
and the related object include data, the data com- 
prising video, picture, sound, and text. 

3. The method of Claim 1 , further comprising: 

(a) enabling a qualification engine to determine 
a context of the indicated object; and 

(b) employing the context of the indicated ob- 
ject to automatically create the query for the da- 30 
tabase. 

4. The method of Claim 1 , further comprising: 

(a) indicating the related object is to be pur- 3S 
chased over a network; and 

(b) providing financial information to an elec- 
tronic commerce service over the network, the 
electronic commerce service enabling the pur- 
chase of the related object when the financial ^ 
information Is complete. 

5. Method for obtaining an Image from a database for 
insertion into a document, the image being related 

to indicated text, comprising: ^ 

(a) enabling an automatic creatkxi of a query 
related to the indicated text, the query having a 
data structure tfiat is recognizable by a search 
engine for the database; so 

(b) providing the query to the search engine, 
the search engine searching the database for 
an Image that is related to the indk^ted text; 

(c) returning the result from the query of the da- 
tabase, the result indicating when at least one ss 
image is related to the indicated text; 

(d) enabling the display of at least one image 
indk:ated by the result from the query, a dis- 



played Image being selectable for insertion Into 
the document; and 

(e) displaying a related image that is inserted 
into the document, so that the related Image is 
associated with the indicated text in the docu- 
ment. 

6. The method of Claim 5, further comprising employ- 
ing a user interface component to automatically cre- 
ate the query for the database, the creatbn of the 
query being related to the indk^ated text. 

7. The method of Claim 6, wherein the user interface 
component enables the indicated text to be edited. 

8. The method of Claim 6, wherein the user interface 
component enables the indicated text to be chosen 
in the document. 

9. The method of Claim 8, further comprising choosing 
the Indicated text in the document for the query of 
the database from within an app\ica\ion program 
that provides the document. 

1 0. The method of Claim 9, further comprising choosing 
a command to create the query for the lndk:ated 
text, the command including an indicatkxi in a con- 
text menu, drop-down menu, pop-up window, dia- 
logue box, toolbar and hot key. 

1 1 . The method of Claim 9, further comprising choosing 
the indicated text in the document for the query of 
the database from an application program that is 
separate from another applicatkxi program that pro- 
vides the document. 

1 2. The method of Claim 5, further comprising enabling 
the indk^ated text to be chosen In another applk:a- 
tion program that does not provkJe the document, 
including a clipboard program. 

13. The method of Claim 6, further comprising: 

(a) enabling a qualificatbn engine to determine 
a context of the indicated text within the docu- 
ment; and 

(b) provkiing the context of the indicated text to 
the user interface component, the user Inter- 
face component empbying the context of the 
indicated text to automatk:ally create the query 
for the database. 

14. The method of Claim 13, wherein the determined 
context of the Indicated text includes text, template, 
sound, video, picture, use and user preference. 

15. The method of Claim 5, wherein the search engine 
is a kx:al search engine, the kx:al search engine 
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employing the query to search for related images 
on a local drive that includes the database. 

16. The method of Claim 15, wherein the local drive in- 
cludes hard disk, floppy disk, tape drive, DVD and 
CD-ROM. 

17. The method of Claim 5, wherein the query's data 
structure includes XML. 

18. The method of Claim 5, further comprising employ- 
ing a network component to communicate with the 
search engine over a network, the network includ- 
ing an intranet and the Internet. 

19. The method of Claim 5, further comprising employ- 
ing a result component to manage the result of the 
query returned by the search engine, the result 
component enabling the parsing, storing and dis- 
play of the result of the query. 

20. The method of Claim 5. wherein the result is in a 
metadata format. 

21 . The method of Claim 5, further comprising request- 
ing at least one related image from the database 
that is indicated by the result of the query, the relat- 
ed image being provided as binary image data. 

22. The method of Claim 21, further comprising em- 
ploying an image component for managing the stor- 
age and display of binary image data. 

23. The method of Claim 21, further comprising em- 
ploying a navigation component to provide at least 
one control for displaying at least one related im- 
age. 

24. The method of Claim 21, wherein the navigatbn 
component enables the display of a group of related 
images in a slide show, the group of related images 
being indicated in the result from the query. 

25. The method of Claim 21, wherein the navigatbn 
component enables another group of related imag- 
es to be displayed in the slide show, the other group 
of related images being indicated in the result from 
the query. 

26. The method of Claim 21, wherein the navigation 
component includes a control for determining a pe- 
riod of time that each related innage is displayed in 
the slide show. 

27. The method of Claim 21, wherein the navigation 
component includes a control for indicating the 
number of the related image in the group that is cur- 
rently displayed in the slide show. 



28. The method of Claim 21, wherein the navigation 
conrrponent includes a control for indicating each 
group of related images that are currently displayed 
in the slide show. 

5 

29. The method of Claim 5, further comprising enabling 
a help facility to provide an explanation of a chosen 
topic and the help facility being launched in re- 
sponse to an input. 

10 

30. The method of Claim 5, wherein the display of at 
least one related image further comprises display- 
ing a thumbnail of the related image. 

31 . The method of Claim 5, further comprises including 
a digital watermark with the related image, the dig- 
ital watermark including information related to the 
content of the related image. 

20 32. The method of Claim 5, further comprises including 
a cookie with the query, the cookie including infor- 
mation related to a user. 

33. The method of Claim 5, further comprising: 

25 

(a) indicating a higher quality image of the re- 
lated image is to be purchased over a network; 

(b) providing financial information to an elec- 
tronic commerce service over the network, the 

^ electronic commerce service enabling the pur- 

chase of the higher quality image when the fi- 
nancial information is complete; and 

(c) inserting the purchased higher quality image 
into the document so that the higher quality im- 

35 age is displayed with the indicated text. 

34. The method of Claim 33, further comprising em- 
pbying an electronic shopping cart to store a poten- 
tial purchase of the higher quality image. 

40 

35. The method of Claim 33, further comprising ena- 
bling a parameter of the higher quality image to be 
indicated. 

45 36. The nnethod of Claim 33, further comprising ena- 
bling a use of the higher quality image to be indicat- 
ed. 

37. The method of Claim 5, further comprising: 

50 

(a) each time indicated text is employed to cre- 
ate the query, automatically saving the indicat- 
ed text as an entry in a history file; and 

(b) enabling at least one of the entries in the 
^ history file to be chosen for the currently indi- 
cated text. 

38. The method of Claim 5, further comprising embed- 
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ding a URL wilh the related image pasted into the 
document, the URL providing a link to information 
associated with the related Image. 

39. The method of Claim 5, further comprising automat- 
ically determining a word that is closest to a cursor 
when no indicated text is provided and employing 
the determined word as indicated text. 

40. The method of Claim 1 8, wherein the network com- 
ponent employs a transmission device to automat- 
ically connect to the search engine for the database, 
the transmissk)n device including cable modem, 
network interface card, telephony nrtodem, satellite 
dish and radio transceiver. 

41. The method of Claim 18, further comprising em- 
ploying the network component to automatically 
connect to the search engine for the datat)ase when 
the search engine is remotely located from the in- 
dicated text. 

42. The method of Claim 1 8, wherein the network com- 
ponent employs a proxy server to automatically 
connect to the search engine for the database. 

43. The method of Claim 1 8, wherein the network com- 
ponent empbys a dial up facility to automatically 
connect to the search engine for the database. 

44. The method of Claim 1 8, wherein the network com- 
ponent launches a browser to automatically con- 
nect to the search engine for the database. 

45. The method of Claim 18, wherein the network com- 
ponent automatically connects to the search en- 
gine. 

46. The method of Claim 1 8, wherein the network com- 
ponent launches a browser to nnanage a purchase 
of a higher quality image associated with the related 
image. 

47. The method of Claim 18, wherein the network com- 
ponent manages a purchase of a higher quality inrv 
age associated with the related image. 

48. A system for obtaining an image from a database 
for pasting into a document, the image being related 
to indicated text, comprising: 

(a) a client process for causing functions to be 
performed, including: 

(i) enabling an automatic creation of a que- 
ry based on the indicated text, the query 
having a data structure that is recognizable 
by a search engine for the database; 



(ii) providing the query to the search en- 
gine, the search engine searching the da- 
tabase for an image that is related to the 
indicated text; 

(iii) enabling a display of at least one relat- 
ed image indicated by the result from the 
query, the display of a related image being 
selectable for insertion into the document; 
and 

(iv) displaying the related image that is in- 
serted into the document, the related im- 
age being associated with the indicated 
text in the document; and. 

(b) a server process for causing functbns to be 
performed, including: 

(i) retuming the result from the query of the 
database, the result indicating when at 
least one image is related to the indicated 
text; and 

(ii) enabling the search engine to kx^te at 
least one image in the database that is re- 
lated to the indicated text. 

49. The system of Claim 48, wherein the client process 
causes further functions to be performed, compris- 
ing: 

(a) indicating a higher quality image of the re- 
lated image is to be purchased over a network; 

(b) providing financial information to an elec- 
tronic commerce service over the network, the 
electronic commerce service enabling the pur- 
chase of the higher quality image when the fi- 
nancial information is complete; and 

(c) inserting the purchased higher quality image 
into the document so that the higher quality im- 
age is displayed with the indicated text. 

50. A computer readable medium having computer-ex- 
ecutable components, comprising: 

(a) a component for automatically creating a 
query related to indicated text, the query having 
a data structure that is recognizable by a search 
engine for a database; 

(b) a component for providing the query to the 
search engine, the search engine searching the 
database for an image that is related to the in- 
dicated text; 

(c) a component for retuming the result from the 
query of the database, the result indicating 
when at least one image is related to the indi- 
cated text; 

(d) a component for enabling the display of at 
least one related image indicated by the result 
from the query, the display of the related image 



IS 



20 



2S 



30 



35 



40 



45 



SO 



11 



21 



EP1 077 414 A2 



being selectable for insertion tnto the docu- 
ment; and 

(e) a component for displaying the related im- 
age that is inserted into the document, so that 
the related image is assodaEted with the indicat- s 
ed text in the document. 

51 . Method for electronicalty purchasing an image over 
a network, comprising: 

10 

(a) automatically creating a query related to an 
indicated object, the query having a data struc- 
ture that is recognizable by a search engine for 
the database; 

(b) providing the query to the search engine, is 
the search engine searching the database for 

at least one image that is related to the indicat- 
ed object; 

(c) returning the result from the query of the da- 
tabase, the result indicating when at least one 20 
image is related to the Indicated object; 

(d) indicating a related image is to be pur- 
chased over the network, financial information 
being provided to an electronic commerce serv- 
ice that enables the purchase of the related im- 2S 
age when the financial information is complete; 
and 

(e) when the related image is purchased, ena- 
bling a display of the related image, so that the 
related image may be associated with the indi- 3o 
cated object. 

52. The method of Claim 51 , wherein the indicated ob- 
ject includes data, comprising video, pk:ture, sound, 
and text. 3S 

53. The method of Claim 51, further comprising: 

(a) enabling a qualification engine to determine 

a context of the indicated object; and 40 

(b) employing the context of the indicated ob- 
ject to automatically create the query for the da- 
tabase. 
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